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ABSTRACT 

Methods were developed for the collection of data with respect to 
the knowledge, skill, and satisfaction of aviators in combat readiness 
training (CRT) . The methods are described and data pertaining to 
aviators flying the T-1A aircraft at the rate of forty-eight hours per 
year is given. 
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I . BACKGROUND 



1.1 THE CRT PROGRAM 

Combat Readiness Training (CRT) is the title of the proficiency 
flying program within the Naval Service, and, as such, is applicable to 
both the Navy and Marine Corps. Based on public law [ 8 ] 1 , executive 
order [9] , and Navy directive [6] , the CRT program has been established 
with the goals, "... to maintain ... basic aeronautical skill . . for 
those aviators and naval flight officers assigned to shore or sea-based 
billets where tactical, operational flying is not required. Without 
such a program, most professional aviators would be, in effect, grounded 
during assignments to ship's company, staff, station, or school duty. 

The CRT program, before recent changes, specified a total of 100 
flying hours per aviator per year with minimum night and instrument fly- 
ing requirements to be accomplished within the 100 hours limit. Beyond 
these specifications, the program has relied on stipulated NATOPS require- 
ments for each aircraft type and local cognizance to provide guidance in 
the conduct of proficiency flying. CRT has been largely a self-taught 
program using obsolescent trainers and used-up operational aircraft, 
operating from bases where staffing and supply support are significant 
problems. At the present time, there are approximately 5300 aviators 
and 1100 Naval flight officers in the CRT program, a program whose 
annual cost is nearly $100 million. 

^refers to document number in bibliography. 



1.2 PROBLEMS IN THE CRT PROGRAM 



With recent military funding cuts and inquiry by the General Account- 
ing Office (GAO), the CRT program has come under close scrutiny. As part 
of the Project 703 military budget cutback, the CRT program was reduced 
to 48 flying hours per year for the second half of FY 70. This temporary 
50% reduction in flying hours was recently extended to all of FY 71 [10] . 
GAO entered the CRT problem when it asked the Secretary of Defense if 
proficiency flying was required for full-time officer students (with 
particular reference to the Naval Postgraduate School (NPS) and NALF 
Monterey) . 

For several reasons, NPS and NALF Monterey became the focal point 
of CRT program study efforts. The GAO question directly involved NPS 
and NALF Monterey, among others; NPS has a current aviator enrollment of 
nearly 500 and NALF Monterey exists solely to provide proficiency flying 
for these aviator students. Further, NPS faculty and students represent 
an in-house source of 'analytical talent' and there were insufficient 
resources in the CNO study program to support another CRT Study. Finally, 
there was a plan to use NALF Monterey to evaluate the use of leased, off- 
the-shelf, modern business jet aircraft in the CRT program. For these 
reasons, NPS was brought into the CRT study effort in January, 1970, and 
became actively engaged in April. 

The principal questions, which ultimately must be convincingly 
answered, are: 
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1. Is the CRT program necessary? 

2. If the program is necessary, what is the optimal program 
structure in terms of hours flown, aircraft employed, use 
of simulators, etc.; and 

3. Is CRT flying required for officers while they are full- 
time students? 

Early study efforts defined a number of problems within the CRT 
program [5] . A basic problem is that the program has no well-defined 
goals or objectives. The stated objective is "to maintain basic aero- 
nautical skill", but this statement lacks precision. For example, a 
typical question is whether basic aeronautical skill can be maintained 
by flying four hours per month, or whether eight hours per month is re- 
quired. It seems clear that no real answer to this question can be 
obtained until "basic aeronautical skill" is further defined in terms 
of specific skills and tasks, until the processes for the development 
and maintenance of these skills and tasks are understood, and until the 
contribution of flying a given aircraft at a given rate, over a given 
time period, and under specific conditions, are determined. 

1 . 3 LITERATURE 

A survey of the literature relating to proficiency flying yielded 
only two recent studies. The LTV Study [27, 28] and the PRC Study [15] 
both indicate that there are no existing, absolute measures of effective- 
ness for the CRT program. In both cases, the kinds of proficiency con- 
sidered were not easily or effectively quantified because they are ill- 
defined. 
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The LTV Study concerned itself primarily with a questionnaire 
administered to a group of aviators which dealt with questions regarding 
retention in the service if the CRT program was abolished. The value 
of the CRT program was then quantified by compiling the cost of training 
replacements for those aviators who indicated that they would resign in 
such circumstances. The main result of the study rests on the subjective 
opinions of a group of aviators not immediately faced with such an occur- 
rence as cancellation of the CRT program. Some have expressed reserva- 
tions about the method and conclusions drawn from this study [34]. 

The study conducted by PRC under the auspices of OSD(SA) addressed 
the problem more rigorously. But PRC assumed initially that CRT flying 
was necessary, and proceeded from there to partially describe the struc- 
ture of an optimal CRT program. 

Additional studies concerning proficiency flying, human factors, 
and related subjects are listed in the annotated bibliography of this 
report . 

1.4 PURPOSE AND SCOPE OF THIS STUDY 

An initial assessment of CRT program study requirements was made 
in May [5]. Attention was then addressed to tasks of more limited scope. 
As noted earlier, during the second half of FY 70 CNO (OP-56) was making 
plans for the use of leased, business jet aircraft in CRT flying at NALF 
Monterey. The leased jet aircraft would completely replace the T-1A 
aircraft then at NALF Monterey. The leased jet arrangement represented 
a departure from the traditional CRT program philosophy of using invest- 
ment-free, ex-operational aircraft and was somewhat of an experiment. 
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A reasonable task then was to develop methodology and data by which the 
results of the experiment could be assessed. 

On about 1 May, the authors began development of a study aimed 
toward making comparisons between the T-1A and leased jet as jet pilot 
proficiency aircraft. Because the leased jets were to begin flying on 
1 July, 1970, any measurement of T-1A proficiency flying would have to 
be completed prior to that time. Time was a critical constraint. 

A number of study procedures were devised; the nature of these 
procedures is discussed in the next section of this report. The study 
procedures developed were designed to be applicable to any aircraft or 
level of flying activity. The purpose of this report is to present the 
study procedures developed and report the data collected relative to 
jet aviator proficiency flying in T-1A aircraft at a rate of four hours 
per month. 
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II. THE STUDY 



2.1 INTRODUCTION 

Little time was available in which to develop the tools for evalu- 
ation. Supposition was that on 1 July the T-1A aircraft would be re- 
placed by the leased jet and that the program would revert to flying 
100 hours per year. It had to be assumed then that if data was to be 
gathered on the proficiency of aviators flying the T-1A at four hours 
per month, it would have to be completed on or before 1 July. 

In evaluating the leased jet as a CRT aircraft, cost, scheduling, 
and maintainability data could be accumulated from existing reporting 
procedures. The more difficult aspects of evaluation include proficiency 
gained or maintained, and aviator satisfaction. It has earlier been 
mentioned that proficiency measurement is most difficult due to lack of 
definition, and that no methods are now available for determining abso- 
lute measures of proficiency. 

It was recognized at an early stage that any measurements of pro- 
ficiency and satisfaction that could be obtained would be only relative 
measures. The overall philosophy of the study effort became that of 
developing relative measures of proficiency, and then, by exercising 
control over conditions and aviators, applying the measures in June and 
again six months later. Any statistically significant differences in 
the measures would then be attributed to differences in aircraft and/or 
flight hours per month. 
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2.2 SOURCES OF DATA 



It was determined that the following sources of data were available 
and that if data were collected from them it would be possible to deter- 
mine if any significant differences were created by differences in air- 
craft flown or the hours per month flown. 

a) The Pilots . This is the most logical point to begin 
measurement of flight proficiency and satisfaction. Ex- 
perience data and measures of flight path vector control 
and instrument procedures appeared to be the most fruit- 
ful study areas. 

b) NALF Monterey . The operations duty clerk, weather fore- 
casters, GCA officer, and operations duty officer are all 
in a position to observe the behavior of CRT pilots and 
report various errors that could exhibit different charac- 
teristics coincident with changes in flight activity and/or 
aircraft. The schedules officer maintains records on the 
number of flights scheduled, flights cancelled, and the 
reasons for cancellations. If the level of flight activity 
or type of aircraft flown has any effect on the level of 
pilot motivation, it should be observable in the cancella- 
tions data. Another possible source of pilot proficiency 
measures is the maintenance office. It is known, for 
instance, that the rate of blown T-1A tires per landing 
has increased since the 48 hours per year schedule was 
implemented . 
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c) The FAA . The Monterey Tower approach control, departure 
control, and ground control, as well as Air Route Traffic 
Control Centers are all in a position to report minor 
infractions of aircraft handling and voice procedures that 
are not of such serious nature as to require formal dis- 
ciplinary reporting, but which could be used to differen- 
tiate between flying activity levels. 

d) Simulators and Operational Flight Trainers . Both devices 
have been shown to indicate the level of pilot proficiency. 

e) Observers . An observer in a flight situation could provide 
valid information on the level of proficiency possessed by 
a pilot, given objective measures of proficiency and control 
of bias between observers. 

f) Flight Recorders . Twenty-four channel data recorders have 
been used to measure flight path vector control parameters, 
and close adherence to established norms is an indication 
of the level of pilot proficiency. 

Given the time constraint, the complete lack of research funds, and 
the reluctance or inability of some of the potential data sources to 
provide data to the study group, the only data sources actually employed 
were the pilots, NALF Monterey, and Flight Observers. 

2.3 STUDY PROCEDURES 

2.3.1 Introduction 

The study procedures consisted of the measurement of a number 
of indicators of pilot knowledge, proficiency, and satisfaction. All 
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measurements are relative rather than absolute. A sample of jet aviators 
at NPS was created in such a way that all pilots studied in June and then 
transitioned to another aircraft or activity level could again be studied 
six months later. 

The sample consisted of all aviators assigned as students to 
NPS who performed their CRT flying in the T-1A, and who were programmed 
to graduate in December 1970, or later. In an attempt to control, at 
least partially, the number of variables and to be able to attribute 
any observed differences to either aircraft type or activity level, the 
sample was further subdivided into five groups as follows: 

a) Group One. Those jet pilots who arrived at NPS prior to 
December, 1969, and would graduate (leave CRT at NALF 
Monterey) in December, 1970; 

b) Group Two. Those pilots who arrived in January, 1970, 
and who would graduate in December, 1970; 

c) Group Three. Those pilots who arrived prior to December, 
1969, and who would graduate in or after June, 1971; 

d) Group Four. Those pilots who arrived on or after 1 
January, 1970, and who would graduate in or after June, 
1971. 

e) Group Five. Those pilots selected randomly from Group 
Four to act as data recorders on the data flights, and 
who would remain at NPS during the entire test period. 

The five groups included a total of 120 jet pilots, broken down by 
group as follows: Group One, 24 pilots; Group Two, 4 pilots; Group 
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Three, 32 pilots; Group Four, 35 pilots; and Group Five, 25 data recor- 
ders. All pilots were flying at a nominal rate of four hours per month, 
actually flown in three hour segments every three weeks. It is assumed 
that such possible relevant variables as age, initial proficiency, num- 
ber of CRT tours, last flying duty, expected time and type of next fly- 
ing tour, etc. are not correlated to either graduation date or first 
letter of last name and that, therefore, the assignment to one of these 
groups is random. However, biographical data of this type was captured 
and future analysis will be possible but will not be specifically 
addressed at this time. Forty-five percent of the sample pilots were 
Lieutenants or Captains, 39% were Lieutenant Commanders or Majors, and 
16% were Commanders. Only 3 of the 120 pilots were Marines, all others 
were Navy. Total flight hours and total jet hours for the data pilots 
(groups one through four) were obtained from NPS Flight Office records. 
The data pilots had accumulated an average of 2420 hours and the range 
of hours was from 778 to just over 5300. The jet hours data was similar 
though with smaller magnitudes. They averaged 1850 jet hours with a 
range of from 241 hours to 4070 hours. 

What follows are descriptions of the four types of knowledge, 
proficiency, and satisfaction data gathering devices employed. 

2.3.2 The Questionnaire 

All sample pilots were convened at the start of our data- 
gathering efforts. They were briefed on the purpose of the study, the 
extent of their participation, and the need for their cooperation. They 
were also told that in any measurement of their knowledge, proficiency, 
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or satisfaction, their ananymity would be preserved. Finally, they were 
cautioned against doing anything which would bias the data gathered. 

After the briefing, all pilots were asked to complete a 
three page questionnaire which dealt with their qualifications and pre- 
vious experience, and their assessment of the T-lA aircraft. Page two 
of the questionnaire sought structured assessments on a standardized 
form; page three provided an opportunity for the pilots to give un- 
structured, personal evaluations of the aircraft. The questionnaire, 
given in Appendix A, was administered to all groups of pilots. 

2.3.3 The Instrument Exam 

Following the completion of the questionnaire, pilots from 
groups one through four were given an unannounced, closed book instru- 
ment examination. The exam was the November, 1969, revision of the 
standard Commander Naval Air, Pacific instrument examination. Forty- 
five minutes were allowed for the exam. The purpose of the exam was to 
obtain a measure of pilot knowledge of instrument procedures. The ex- 
amination is not given in an appendix of this report since it represents 
proprietary information. 

2.3.4 NALF Monterey Operations Data 

Data on flight operations at NALF Monterey were collected to 
provide information relative to pilot knowledge and motivation. The 
duty forecaster and the operations duty clerk were given forms on which 
to record errors made in the filing of DD-175 forms. The study forms, 
given in Appendix B, were to be completed for every T-lA flight flown 
in the period from 22 May, 1970, to ] August, 1970. The GCA officer 
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was also provided with data forms, but the GCA unit was down throughout 
almost the entire study period due to antenna replacement. 

The scheduling officer provided data on flights scheduled, 
flights actually flown, and information on cancellations. The cancella- 
tion data was to be used as a measure of pilot motivation. Once again, 
it is pointed out that all measurement of pilot knowledge, proficiency, 
or satisfaction were relative rather than absolute. The measurements 
become useful when compared to another set of measurements taken on the 
same aviator group but at a different flight activity level or in a 
different proficiency aircraft. 

2.3.5 Data Flights 

This report is about proficiency flying, and, while it is 
felt that all measurements described previously are related to proficiency 
flying, they were not direct measures of flying proficiency. Since 
direct measurement of flying skill was deemed imperative and since 
neither sophisticated simulators nor instrumented aircraft were available, 
the program of data flights with "back-seat" data recorders was conceived. 
A standardized 1.5 hour flight was developed. The pilot whose proficiency 
was being measured, the data pilot, commanded the aircraft and flew as 
though it were a solo flight. The data recorder in the back seat of the 
T-1A monitored the data pilot from preflight through shutdown. 

More than 100 scores in sixteen categories were given to the 
data pilot by the data recorder in a data flight. The basis for these 
scores was a data flight book with sequentially arranged questions, cri- 
teria for awarding grades, and space to record the grade awarded. A 
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copy of the data flight book format is shown in Appendix C. The book 
itself was reduced to 5 x 8 inch size, printed on card stock, and spiral 
bound for the data recorder’s kneeboard. 
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III. STUDY RESULTS 



3.1 PREFACE 

The study results are a compilation of the data obtained using the 
measures or measurement devices described in the previous section. The 
data are relative indicators of pilot knowledge, skill, and satisfaction. 
Few conclusions, if any, can be drawn from the data by itself. Since 
all measures employed are relative, conclusions can only be drawn between 
sets of similarly-obtained data. No appeal to absolute measures is 
possible . 

3.2 THE QUESTIONNAIRE 
3.2.1 Page One 

The questionnaire is given in Appendix A. Page one of the 
questionnaire dealt with pilot experience data. The data are the sums 
of the responses made by each aviator without alteration. Each question 
is discussed in turn. 

Question 3 . This question categorizes aircraft presently 
being flown by the sample aviators, and includes aircraft being flown 
for proficiency at NALF Monterey. Also, the totals by category include 
multiple counts for those pilots flying both the T-1A and the US-2 at 
NALF Monterey, plus those flying at other air stations. One sample 
pilot is flying exclusively at another air station. The breakdown is 
as follows: 
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Aircraft Flown 



Number of Pilots 



T-1A 



125 



US-2 



3 



A4/TA4 



3 



T-28 



1 



Multiple 



6 



Question 4 . NATOPS qualification in type of aircraft pre- 



sently flying, includes all the aircraft being flown by aviators at NPGS 
and elsewhere as noted above. 



qualifications as NATOPS evaluation or maintenance inspection pilot at 
NALF Monterey or elsewhere. It is noteworthy that the expected alterna- 
tive qualification of assignment to the instrument flight check board 
was not included. A "Yes" response means that the particular aviator 
is qualified as either or both NATOPS or test pilot. "No” means not 
NATOPS, and not test pilot. The remaining responses are self explanatory. 



NATOPS Qualified 96 



Not NATOPS Qualified 31 



Question 5 . This question is a statement of individual 



Yes - 18 
No - 98 
Both - 4 



NATOPS (only) - 4 (A4) 

Test (only) - 1 

Inst (only) - 1 
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Question 6 . The categories of type aircraft last flown by 



the aviator sample includes in several cases multiple types, and aggre- 
gates by basis model, e.g., A-4 includes TA4F, A4E, A4F, A4B, and A4C. 
Two responses not included in the category counts were responses as to 
the last time the individual had flown, chronologically, e.g., the 
previous day. 



Aircraft Type Number of Aviators 



A4 


46 


A7 


24 


F4 


24 


F8 


11 


A6 


9 


T-28 


5 


RA-5 


4 


F9 


4 


T1 


3 


T2 


3 


A3 


2 


P3 


2 


T33 


1 


T39 


1 


TF10B 


1 


E2 


1 


P2 


1 


C45 


1 


Cl 


1 


C131 


1 


R4D 


1 


C 


117 


US2 


1 


OV-IO 


1 
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Question 7 . Estimated flying hours logged in year prior to 
NPGS, is organized so it may be used in a histogram with the intervals 
specified. The intervals themselves include the lower endpoint but do 
not include the upper endpoint. In three cases, the aviators responded 
by giving an estimate of their total flight hours accumulated before 
reporting to NPGS, from the time they first began flying. Those re- 
sponses are not included in the tabulation. It should be emphasized 
that these are estimates and in all cases are round-offs to the nearest 
5, 10, 50, or 100 hours. 



Hours 



Number of Aviators 



Less than 50 

50-100 

100-150 

150-200 

200-250 

250-300 

300-350 

350-400 

400-450 

450-500 

500-550 

550-600 

600-650 

650-700 

700-750 

750-800 

800-850 

850-900 

900-950 

950 and greater 



0 

2 

7 

12 

15 

27 

30 

15 

6 

3 Mean 300.4918 
2 
1 
0 
0 
0 
0 
1 
0 
1 
0 
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Question 8 . This question asks if an aviator’s last tour 
prior to NPGS was in a CRT billet. It needs no further explanation, 



Yes - 6 

No - 115 

Question 9 . If last tour was CRT, what was the tour dura- 
tion in months? This question had the below listed responses, and, 
again, the interval endpoints are defined as in question 7: 

Months Number of Aviators 

0-12 2 

12-24 3 

24-36 1 



3.2.2 Page Two 

Page two of the questionnaire asked for assessments of the 
present jet CRT aircraft. The actual responses are given in Table 1. 

The data gathered on the T-1A CRT characteristics, which were the 
opinions expressed by 120 aviators within the sample, indicated that 
the T-1A was considered an average to poor aircraft for CRT purposes. 
Although general T-1A procedures were considered to be average or above 
by 86% of the pilots sampled, their overall impressions of the safety 
of the aircraft indicates only 61% of them felt safe in the aircraft. 

The greatest dissatisfaction seemed to be with the short endurance char- 
acteristic of the T-1A (91%) and the comfort factor (73%). 
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TABLE 1. OPINIONS ABOUT T-1A CRT CHARACTERISTICS 



1 . 


Procedures Great 


Good 


OK 


Poor 


Lousy 


N/A 




a. Preflight 


3 


48 


60 


7 


2 


0 




b. Engine start 


2 


49 


54 


12 


3 


0 




c. Emergency 


i 


36 


59 


18 


6 


0 




d. Post flight 


2 


33 


67 


11 


6 


1 


2. 


Handling 
a. Taxi 


1 


5 


28 


33 


50 


4 




b. Take-off 


3 


26 


72 


15 


4 


0 




c. Inst. Flt-Simulated/ac tual 


6 


39 


37 


27 


10 


0 




d. Climb 


0 


23 


62 


28 


7 


0 




e. Cruise 


0 


21 


54 


32 


13 


0 




f. Descent 


1 


24 


76 


15 


4 


0 




g. Approach 


2 


38 


54 


19 


7 


0 




h. Landing 


2 


31 


47 


27 


13 


0 




i. Slow flight 


5 


37 


62 


13 


2 


1 


3. 


Safety 

a. Overall impression 


0 


24 


49 


33 


10 


0 




b. Survival equipment 


2 


19 


69 


20 


9 


1 




c. Escape systems 


4 


27 


50 


31 


6 


1 




d. Stall warning 


10 


44 


56 


6 


3 


1 




e. Stall/spin recovery 


22 


51 


37 


3 


2 


5 


4. 


Navigation equipment 


13 


41 


35 


19 


8 


3 


5. 


Endurance 


0 


0 


10 


37 


72 


1 


6. 


Aerobatics 


3 


26 


60 


23 


7 


1 


7. 


A/C flight limitations 


2 


15 


57 


27 


8 


1 


8. 


Availability (maintenance) 


0 


13 


40 


53 


14 


0 


9. 


Support away from home 


1 


9 


35 


31 


34 


10 


10. 


A/C reliability 


0 


12 


53 


40 


13 


2 


11. 


Comfort 


0 


2 


28 


39 


49 


2 


12. 


Convenience 


2 


7 


48 


40 


21 


2 


13. 


How good is the A/C as a CRT A/C? 


0 


16 


43 


46 


15 


0 
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3.2.3 Page Three 



Page three of the questionnaire offered the sample aviators 
the opportunity to express opinions about the present jet CRT aircraft 
and program, without the highly structured format of page two of the 
questionnaire. Their responses, with and without attempts to summarize 
and categorize them, are now given. 

The first half of page three, question II, responses were 
summarized in Table 2, followed by exaplanat ions of the categories used 
to classify the responses. 

TABLE 2. SUBJECTIVE OPINIONS ABOUT T-1A 

Qualified Qualified No 







Yes 


Yes 


No 


No 


Comment 


a . 


Confidence 


54 


38 


7 


25 


8 


b. 


Comfort 


25 


4 


76 


17 


5 


c . 


Fun to Fly 


49 


24 


30 


10 


5 


d. 


Maintains Skills 


23 


62 


7 


25 


6 



1. Yes and No . These categories include only the number of direct 
opinions recorded on the questionnaire, not carrying any modifying 
remarks, by each sub-category. 

2. Qualified Yes . This category includes all favorable opinions by 
question, with modifying remarks as follows: 

a. Confidence . Includes remarks about T-1A systems, engine power, 
aircraft age, maintenance quality, 4 hours per month flying time 
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restriction, aircraft endurance characteristics, IFR/VFR weather capabil 
ities, ground training quality, and scheduling policies. 

b. Comfort . Includes remarks about cockpit elbow room, the Martin 
Baker Seat particularly and in general, leg room in both cockpits, com- 
parisons between cockpits, ability to reposition rudder pedals, seat 
height relations to pilot’s head height, and comparisons with the A4 . 

c. Fun to fly had specific remarks about poor endurance, poor per- 
formance capabilities, 4 hours per month limitations, time between 
flights being excessive, acrobatics capability of T-1A, and loss of 
familiarity with aircraft. 

d. Maintains skills question had remarks about IFR versus VFR fly- 
ing, acrobatic capability, T-1A good for instruments but nothing else, 
skills retained were not operational type skills, skills slowly eroding, 
not safe, aircraft systems considerations, the 4 hours per month re- 
striction versus 8 or 12 hours per month, and the T-1A landing character 
ist ics . 

3. Qualified No is defined as the qualified Yes, and included: 

a. Conf idence . Remarks about aircraft age, systems, maintenance, 
ground training, checkouts, scheduling policies, and the 4 hours per 
month restriction. 

b. Comfort included marginal leg and head room of cockpit, and 
other items considered unsafe. 

c. Fun to fly had remarks about aircraft endurance, poor aircraft 
performance, satisfies urge to fly, acrobatic capability, 4 hours per 
month restriction. 
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d. Maintains skills had many remarks such as 4 hours per month re- 



striction, aircraft capabilities and performance not a challenge, com- 
parisons to operational fleet aircraft and operational missions, and 
skills retained not being the necessary operational skills. 

4. No comment was the number of the various questions left blank or 
marked as not considered applicable. 

5. It should be noted that the above data was collected as specific re- 
marks about the particular subject only. As would be expected, more than 
several aviators presented more than one idea on each subject. Summing 
the figures in Table 2 will not yield the number of aviators questioned. 

The second half of page three, question III, yielded a number 
of interesting comments. The following is a compilation of the salient 
topics and ideas given by this group of aviators when asked to comment. 
The subject is listed, followed by a resume of opinions. The number 
following each subject title signifies the number of aviators presenting 
ideas about that subject. A total of 125 different opinions were re- 
corded from 56 of the 120 pilots polled. No comments were received from 
the remaining 64 pilots. 

1. The CRT Aircraft (23) 

- Aviators should be assigned to fly a CRT aircraft according 
to their previous experience. (1) 

- A CRT aircraft requires an acrobatic capability to retain 
operational skills especially for fighter/attack pilots. (8) 

- T-1A is poor due to performance capabilities. (9) 

- Enjoy the T-1A but would switch to S-2 in preference to a 
civil jet. (1) 
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- T-1A preferable to civil jet (1), perfectly adequate. (1) 

- T-28 or T-2 preferable to T-1A. (1) 

- The T-lA is similar to all fleet aircraft in that it has an 
ejection seat, a bubble canopy, throttle and engine controls 
on the left, stick versus yoke, and acrobatic capability. (1) 

1. CRT Generally (19) 

- CRT is a misnomer. (3) 

- 4 flight hours a month insufficient. (1) 

- 4 flight hours a month hazardous. (1) 

- 4 to 8 flight hours per month will not guarantee anyone's 
skill, but will keep from completely losing that skill and 
will make the return to operational flying less traumatic. (1) 

- Since anyone returning to the fleet receives CRAW/RAG train- 
ing, why not admit that CRT is worthless? (1) 

- Flying 24 hours a fiscal half year makes it hard to do any- 
thing but work hard to stay qualified (in type). (1) 

- CRT flying "worthless" for aviators in "student" billets. (1) 

- CRT flying in poor shape due to people who manage it. (1) 

- (I have) Some fear plus apprehension as to being able to hack 
it in the fleet; hack going aboard ship; can I keep up with 
the first tour types? With more realistic proficiency flying, 

I would not have though twice about returning to the fleet. (2) 

- It is doubtful that fun is the object of flying this type of 
machine or doing this type of flying. A waste of time. (1) 

- CRT in itself is ridiculous. Flying a "lesser aircraft" 
does not keep you proficient in anything [presumably mission 
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proficiency] that is necessary. CRAW/RAG's should be able 
to provide information as to effectiveness of CRT. (1) 

- At 4 hours per month, it takes the whole first half of a 
hop to be sure of what I am doing. (1) 

- Consider general airmanship most important as part of CRT. (1) 

- No reduction in skill or confidence felt when reduced from 8 
to 4 hours per month. (1) 

- Number of flights per unit of time period would be better 
measure of proficiency than flight hours. (1) 

- Consider some training accomplished just by strapping into 
the aircraft. (1) 

3. Scheduling and Airbase Facilities (20) 

- As a [NATOPS, Instrument] check pilot, I never get to fly, 
just sit in back and watch. (1) 

- No transition syllabus into the T-1A. (1) 

- No hood for simulated instruments. (1) 

- Need abort gear at NALF Monterey. (1) 

- I have been scratched or aborted an excessive number of 
times. (1) 

- Need NAMO , emergency procedures ground trainer, periodically 
administered emergency procedures examination, and a cockpit 
checkout before first hop. (1) 

- Cannot understand why [purpose] "A" time should count against 
"B” time maximum total flight hours. (1) 

- Want more variation in type of hops flown. (3) 

- Want more/unlimited cross countries. (1) 
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- Want briefed multiple aircraft flights scheduled. (6) 

- Want 8 to 10 flight hours per month. (1) 

- Want better maintenance including preflights. (2) 

- Runways and facilities at Monterey are marginally safe in 
bad weather. (2) 

3.3 THE INSTRUMENT EXAMINATION 

The unannounced, closed-book, COMNAVAIRPAC instrument exam was 
administered to all data pilots in sample groups one through four. The 
data recorders, group five, did not take the exam. A total of 96 data 
pilots took the exam: one of these was subsequently dropped from the 

sample. The exam was graded in such a way that the maximum score was 
54 points. 

The average score attained by the data pilots was 33.3, with a 
standard deviation of 6.8 points. The breakdown of scores by interval 
was : 



Total Score 


Numbe 


0 - 14.5 


0 


15 - 19.5 


1 


20 - 24.5 


8 


25 - 29.5 


20 


30 - 34.5 


31 


35 - 39.5 


21 


40 - 44.5 


9 


45 - 49.5 


6 


50+ 


0 
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The exam scores indicated that 18% of the pilots tested answered 
less than half of the questions correctly and that the average percent- 
age of correct answers was 67% (33.3/54). These scores, by themselves, 
are not necessarily meaningful. They are scheduled to be used with the 
scores obtained by pilots taking the November 1970 edition of the 
COMNAVAIRPAC instrument exam. The new exam will be administered toward 
the end of this calendar year to members of groups one through four who 
since 1 July have been flying at a nominal rate of eight hours per month. 

3.4 NALF MONTEREY OPERATIONS DATA 
3.4.1 Schedules Data 

Operations data from NALF Monterey was collected during the 
period from 22 May 1970 to 31 July 1970 on every T-1A flight. During 
the period, a total of 412 sorties were scheduled and a total of 369 
sorties were actually flown. There were an even 100 cancellations for 
various reasons, not all of which necessarily cancelled the sortie. 

Full data is shown in Table 3. 

A part of the cancellations data reflects pilot motivation. 
Though not quantified, it is known that pilots are now cancelling for 
weather reasons when they would have flown in the same weather when 
they were on a 100 hours per year level. An interesting point on 
cancellations for academic reasons was found in [12]. The Barton, 
et . al . , study group found that only 30% of the aviators in their first 
CRT tour had ever cancelled a flight due to study work load or other 
academic considerations, while 67% of the aviators with prior CRT tours 
had found it necessary to do so. It is speculated that motivation to 
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TABLE 3 



NALF MONTEREY T-1A SCHEDULES DATA: 22 May - 31 July 1970 



SORTIES SCHEDULED 




412 


Scheduled Sorties Flown 


323 




Non-Scheduled Sorties Flown 


Maintenance 


17 




Purpose B 


21 




Other 


TOTAL SORTIES FLOWN 




369 



CANCELLATIONS 

Weather 22 

Scheduling Error 17 

A/C Went Down 15 

Academic 14 

Pilot(s) Sick 13 

Pilot (s) No-Show 7 

Pilot Request (Other than Academic) 6 
Other 6 

TOTAL CANCELLATIONS 



100 
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participate in CRT flying, rather than academic pressures, is the real 
reason behind "academic" cancellations. 

3.4.2 Duty Forecaster Data 

On every T-1A flight during the 22 May - 31 July 1970 period, 
the duty forecaster maintained a record of DD-175 and/or weather briefing 
deficiencies. This data shows that problems were encountered with 8% of 
the DD-175 f s and/or briefings. Table 4 gives a breakdown of this data. 



TABLE 4. CRT STUDY: DUTY FORECASTER TALLY 



TOTAL FLIGHTS FLOWN 369 

Information on DD-175 Incomplete 20 

Alternate Requirements Not 

Understood 4 

Inattention to Brief or Did 

Not Understand 1 

Other 5 

TOTAL PROBLEMS 30 



3.4.3 Operations Duty Desk Data 

Again, all T-1A flight DD-175 forms were monitored for 
problems of various types. The "problem rate" at the operations duty 
desk was 43%, as detailed in Table 5. 
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TABLE 5. CRT STUDY: OPS DUTY DESK CLERK 



TOTAL FLIGHTS FLOWN 



369 



Incorrect DD-175 



58 



Incomplete DD-175 



71 



Misunderstanding over IFR/VFR 2 



Needed help 



14 



Other 



15 



TOTAL PROBLEMS 



160 



3.5 DATA FLIGHTS 

The data flights were begun at NALF Monterey on 8 June 1970, and 
completed on 21 August 1970. Early problems included the three-week 
voluntary sign-up schedule in effect at the end of the NPS academic year, 
and, after 1 July, scheduling difficulties. The scheduling problems 
were satisfactorily resolved ultimately. 

The data recorders, group five, were assigned from two to six data 
flights and CRT maximum hour limits were waived where necessary. All 
data recorders were briefed on the purpose of the flights, and the data 
flight book, page by page. Further, they were asked not to show the 
data flight book to the data pilot and not to discuss the results with 
him. The data flight book is shown in Appendix C. 

Differences between data recorders exist, as well as differences in 
how a given data recorder interprets the grading criteria from flight 
to flight. The data flight scores awarded were relative scores. In 
an attempt to minimize the subjective aspects of data pilot evaluation, 
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the study was designed so that the next series of data flights will be 
conducted utilizing the same data pilots and the same data recorders. 

For completeness, it is recorded that some problems were encountered in 
the program of data flights, that the problems involved both data pilots 
and data recorders, and that the problems disappeared when positive 
leadership was provided by NALF Monterey Operations. 

The complete results of the 96 data flights are given in Appendix D; 
there were 96 data flights but one of the data pilots was later dropped 
from the sample. The data was first summarized in terms of the scores 
achieved in each of the 16 flight phases, from preflight to shutdown. 

That data was then aggregared into six categories as follows: 

Category Question 



1 . 


Preflight 


1.2 


2. 


Fundamentals 


3,4,14,15 


3. 


Climbs 


5,9 


4. 


Enroute 


6,10 


5. 


Letdowns 


7,8,11,12,13 


6. 


Other 


16 



Computation of average scores, etc., is difficult since the maximum 
possible score could differ from flight to flight. Examples were that 
the entry for landing in Monterey could have been under VFR conditions 
(question 12) or IFR conditions (question 13) but usually not both al- 
though some data recorders requested the data pilot to do both when 
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possible. A further example is that several data flights were completed 
with a hot refueling cycle and therefore shutdown (question 15) was not 
applicable . 

As this set of data flights are to be compared to a set of data 
flights later this year, and since this data cannot "stand by itself", 
little analysis of the data was done at this time. The maximum possible 
score was 97 points. The range of data flight raw scores obtained was 
from -38 to +91. The average raw score was 32.8 points, in spite of the 
fact that the scoring criteria were designed in such a way that the 
average aviator should, theoretically, have received a zero score. 

In aggregating the scores into six categories, average scores were 
computed taking into account the maximum score possible for each individual 
data flight. The maximum possible score varied from flight to flight as 
explained above. In computing average scores by category, the total 
points achieved were divided by the maximum possible, flight by flight. 

The results were as follows: 



Category 


Average Score 


1. Preflight 


.438 


2. Fundamentals 


.468 


3. Climbs 


.177 


4. Enroute 


.384 


5. Letdowns 


.422 


6. Other 


.477 
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The testing of the proficiency pilots was conducted within the pro- 
ficiency flying context. That is, the data flights did not involve 
tactics, high-g maneuvers, or carrier landings. Further, the data flight 
data does not provide absolute measures of proficiency. Even so, it is 
interesting to note the relative proficiencies as indicated by the cate- 
gorized data flight average scores. Climbs, enroute segments, and let- 
down are performed with relatively less proficiency than, for example, 
fundamentals . 

From data flights to be conducted later this year, after the sample 
data pilots have flown at the four hour per month rate for a longer time 
(one year), it should be possible to indicate the types of skill deterior- 
ation caused by the four hour per month schedule, if, in fact, there are 
any . 
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IV. SUMMARY AND CONCLUSION 



This report has been addressed to the study of proficiency flying 
within the Navy, and specifically, to the task of developing methods 
whereby changes in the CRT program (aircraft, or hours flown primarily) 
can be evaluated with respect to aviator knowledge, skill, or satisfaction. 

Measures or measurement techniques have been developed and applied 
to a sample of 96 jet aviators in the NPS/NALF Monterey community. In 
proficiency, definition, let alone measurement, is difficult. None of 
the data taken represents measures from an absolute scale. The data 
described in the previous section is useful only when compared to data 
obtained in a similar manner with a modified CRT program structure. 

As indicated in the introduction, the study was conceived in the 
context of evaluating the anticipated change from T-1A leased jet air- 
craft. The 1 July 1970 change-over to leased jets did not take place. 

In its place, NALF Monterey has received authorization to schedule 40 
aviators, from groups one through four, for 100 hours per year for FY 71. 

In December 1970, the "100 hour group" will again be subjected to the 
questionnaire, the instrument exam, NALF Monterey Operations scrutiny 
of their DD-175 r s, and the data flights. When this analysis is completed, 
it should be possible to make statements relative to the knowledge, skill, 
and satisfaction of jet CRT aviators flying 100 hours per year as com- 
pared to those aviators flying 48 hours per year. 

It is further anticipated that the Aviation Training Division (OP-56) 
will follow the 40 "100 hour aviators" in their subsequent CRAW/RAG 
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billets in an attempt to determine whether the CRT flying rate has an 
influence on retraining requirements, costs, or time. 
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APPENDIX A 

SAMPLE PILOT QUESTIONNAIRE 
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Pilot Questionnaire 



CRT STUDY 



1. File Number 

2. Graduation date (month, year)? 

3. What A/C are you presently flying? 

4. Are you NATOPS qualified in this A/C? 

5. Are you a NATOPS check pilot and/or a 

maintenance test pilot? 

6. Type A/C last flown? 

7. Estimated pilot hours logged in year 

prior to NPS 

8. Was last assignment prior to NPS in 

CRT flying (yes, no)? 

9. if question (8) affirmative, how long 

was the CRT tour (months)? 
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I. T-1A CRT CHARACTERISTICS: 



Please indicate your opinion of the T-1A as a CRT A/C: 



1. 


Procedures 




a. 


Preflight 




b. 


Engine start 




c . 


Emergency 




d. 


Post flight 


2. 


Handling 




a. 


Taxi 




b. 


Take-off 




c . 


Instrument Flight 
Simulated /Actual 




d. 


Climb 




e . 


Cruise 




f . 


Descent 




8- 


Approach 




h. 


Landing 




i . 


Slow flight 


3. 


Safety 




a . 


Overall impression 




b. 


Survival equipment 




c . 


Escape systems 




d. 


Stall warning 




e . 


Stall/spin recovery 



4. Navigation equipment 

5. Endurance 

6. Aerobatics 

7. A/C flight limitations 

8. Availability (maintenance) 

9. Support away from home 

10. A/C reliability 

11. Comfort 

12. Convenience 

13. How good is the A/C as a 

CRT A/C? 



Great Good OK Poor Lousy N/A 
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II. Please indicate your personal, subjective feelings about the T-1A 



with regard to: 

1. Confidence 



2. Comfort 



3. Fun to fly 



4. How well does flying the T-1A maintain your flying skills 



III. Additional Comments - if you care to 



38 



APPENDIX B 

NALF MONTEREY OPERATIONS 
DATA COLLECTION FORMS 
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NALF SCHEDULES DATA LIST 



CRT STUDY 
*Daily Summary 

Date 

T-1A Leased Jet 

Sorties Scheduled 

Scheduled Sorties Flown 

Non-scheduled Sorties : 

Maintenance 

Purpose B 

Other 

Cancellations: 

Academic ______ 

Pilot (s) sick 

A/C went down 

Non-availability of A/C 

Pilot (s) no show 

Other (specify) 



Total 
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DUTY FORECASTER 



CRT STUDY 

Please fill out the following information for every T-l or leased jet 
D D-175 filed during your watch. Turn in all completed forms to the 0D0 . 



Pilot f s file no. 



Copilot's file no. 



Date 



Information on DD-175 
incomplete 



Inattention to brief or 
did not understand 



Alternate requirements 
not understood 



Other (specify) 
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OPS DUTY DESK CLERK 



CRT STUDY 

Please fill out the following information for every T-l or leased jet 
DD-175 filed during your watch. Turn in all completed forms to the 0D0 . 



Pilot's file no. 



Copilot's file no. 



Date 



Incorrect DD-175 Needed help 



Incomplete DD-175 Other (specif y) 

Misunderstanding over 

IFR/VFR None 
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MONTEREY GCA 



CRT STUDY 

This is a request for data for a study of T-1A and leased jet pilot pro- 
ficiency. An entry on this form does not put pilot on report. Please 
record and briefly but specifically describe each occurrence by a T-l 
or leased jet crew of glitches; i.e., procedural error, violations of 
SOP or local regulations, poor voice procedures, confusion, interfer- 
ence with traffic, etc., as you may feel appropriate. 



CALL SIGN TIME 



T-1A and LEASED JET 

DATE GLITCH 
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APPENDIX C 
DATA FLIGHT BOOK 
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CRT STUDY DATA FLIGHT 



Data Pilot file no. 
Data Recorder file no. 
Bu No 
Date 



DATA FLIGHT SPECIFICATIONS 

1.5 hours first pilot time for data pilot 

IFR to other than home field (enter PCA if feasible) 

Penetration/CCA/missed approach at IFR intermediate destination 
(Lemoore ) 

Return to Monterey (VFR if VMC) (or other destination) 
INSTRUCTIONS TO DATA RECORDER 

Circle appropriate score. If not applicable, circle zero. Add 
remarks if desired. Do not brief or debrief data. Do not discuss 
or show forms to data pilot. 
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1. PLANNING AND FILING 

a. DD-175 : 

Completed correctly without reference to FLIP +1 

Correct with reference to FLIP 0 

Incorrect upon submission to 0D0 -1 

b. Weather: 

Understood, asked pertinent questions, checked charts +1 

Passive, but understood 0 

Failed to understand, missed significant item, or didn't 

know alternate requirements -1 

c. Planning Parameters: 

Used flip lists and correct TAS , IAS and fuel flow, 

range and time to climb +1 

Used accurate estimates for above 0 

Guessed or failed to plan above -1 

d. Familiarity with Publications (FLIP, Enroute, Supplements, 

Terminal pubs and NATOPS check list): 

Knew where to find and how to use desired information +1 

Knew what he wanted but not where to find it 0 

Failed to use or misinterpreted pubs -1 

2. PREFLIGHT 

a. Knowledge of Inspection Points: 

Complete knowledge of points and tolerances +1 

Overlooked one major point 0 

Incomplete knowledge, overlooked more than one -1 

b. Thorough or Cursory Inspection: 

Thorough, concentrated +1 

Minimal 0 

Cursory or lackadaisical -1 

c. Ejection Seat/Pins: 

Knew all points and tolerances; thorough inspection +1 

Satisfactory, minimal effort 0 

Marginal, would have missed on unsafe situation -1 

d. Use of checklist: 

Did, thoroughly and correctly +1 

Didn't (didn't need it) 0 

Didn't use it (should have) -1 
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3. START AND TAXI 

a. Use of Checklist: 

Did, thoroughly and correctly +1 

Didn't (didn't need it) 0 

Didn’t use it (should have) -1 

b. Malfunction Procedures (if applicable): 

Malfunction occurred, handled properly +1 

No malfunction 0 

Goofed -1 

c. Post-start Checks and Signals: 

Known, understood, done properly +1 

Minor, but no unsafe, errors 0 

Not known or not understood or improper -*1 

d. Taxi Voice Procedures: 

Proper radiotelephone procedures +1 

Satisfactory, but minor errors 0 

Arrogant, abusive, intolerant, or major error -1 

e. Taxiing Technique: 

Proper power, speed, steering and configuration +1 

Satisfactory, but rough 0 

Imprudent or improper technique -1 

f. Systems Checkouts: 

Correct and thorough pretakeoff checks +1 

Some uncertainty, but safe 0 

Incomplete or incorrect -1 

g. Instrument Clearance: 

Clear, concise, correct readback +1 

Minor problem, but correct 0 

More than one misunderstanding, or sloppy -1 



4. TAKEOFF 

a. Runup and Final Checks: 



All systems on, checks quick, sure and complete +1 
Checked most systems, monitored engine performance 0 
Forgot more than one system or rolled with major discrepancy -1 

b. Takeoff Directional Control: 

Smooth, no significant deviations from centerline +1 
Minor deviations, reasonable corrections 0 
Swerved, rough -1 

c. Liftoff and Cleanup: 

Rotated at proper speed, smooth, correct pitch +1 
Minor deviations, observed airspeed restrictions 0 
Rough, cleaned up too early or too late -1 

d. Heading and Attitude Control: 

Smooth and sure (+ 3 degrees) +1 
OK (3-8 degrees) 0 
Rough, large deviations -1 
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5. CLIMBOUT AND DEPARTURE 

a. Climb Schedule: 

+ 10 KIAS, smooth +1 

+ 15 KIAS 0 

> 15 more than once, or rough -1 

b. Adherence to Departure Clearance: 

(indicate which SID used) Bay One [ ] Cannery One [ I 

No errors (+ 5 degrees), (4- 100 ft.) +1 

Minor errors (5-10 degrees), (+ 200 ft.) 0 

Major error (s) (busted clearance, missed check point) -1 

c. Climb Safety Checks: 

Done, correctly, timely -Hi 

Done, perfunctorily 0 

Missed or mistaken -1 

d. Navaid Usage : 

Checked ID, correct selection, IFF +1 

Minor errors, but made proper path 0 

No ID check, selectors wrong, missed freq. change -1 

e. Radial Course Control During Transition Climb: 

+ 2 degrees -Hi 

+ 2-5 degrees 0 

+ > 5 degrees -1 

f. Arrival at Assigned Altitude/FL: 

-H 100 ft. and smooth -Hi 

100-200 ft. 0 

+ > 200 ft. or rough -1 

g. Voice Radio Procedures: 

Clear, concise, timely, sharp -Hi 

Proper 0 

Confused, incorrect or untimely -1 
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6 . ENROUTE 

a. Clearance Adherence: 

Did +1 

Minor errors 0 

Didn’t -1 

b. Course Control: 

4 - 2 degrees average, not more than 5 degrees +1 

Never more than 10 degrees off 0 

> 10 degrees -1 

c. Altitude Control: 

Generally + 50 ft., occasional 4 - 100 ft. +1 

4 - 100 ft., never more than 200 ft. off 0 

4 - > 200 ft. -1 

d. Basic Airwork: 

Very smooth, always in control 4-1 

Generally smooth 0 

Improper corrections, rough, or both -1 

e. Headwork and Anticipation: 

Always thinking, checking, planning ahead 4-1 

Occasional lapses 0 

Generally behind -1 

f. Navigation and Navaid Usage: 

Proper switchover points, crosschecks, ID 4-1 

Conformed to clearance/airway, minor errors 0 

Drifted off airway, skipped ID, etc. -1 

g. Fuel/Oxygen Systems Monitor /Management : 

Did, properly and continually 4-1 

Minor lapses, intermittent 0 

Inattention or worse -1 

h. Lookout Doctrine: 

Head up and out, systematic search 4-1 

Looked out sometimes 0 

Head in cockpit -1 

i. Ask the pilot, "If you had a major malfunction right now, 
what would you do and where would you go?" 

Immediate, adequate and appropriate response 4-1 

Had a good idea what and where within 10 seconds 0 

Inadequate and/or indecisive -1 
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7. ARRIVAL AND PENETRATION 

a. Headwork and Anticipation, Descent Checklist: 

Had previewed, fuel/weather checks, planned descent +1 

Responded properly, but not well ahead, and not behind 0 

No anticipation, got behind -1 

b. Basic Airwork: 

Very smooth, always in control +1 

Generally smooth 0 

Improper corrections, rough, or both -1 

c. Radial/Arc Course Control: 

4 2 degrees and 4 1 mile, occasional deviation +1 

4 2-5 degrees and 4 1-2 miles, occasional deviations 0 

Frequent >45 degrees and/or > 2 miles -1 

d. Arrival at Assigned Altitude: 

4 100 ft. and smooth +1 

100-200 ft. 0 

+ 200 ft. or rough -1 

e. Voice Radio Procedures: 

Clear, concise, timely, sharp +1 

Proper 0 

Confused, incorrect or untimely -1 

f. Basic Airwork: 

Very smooth, always in control +1 

Generally smooth 0 

Improper corrections, rough, or both -1 

g. Clearance Adherence: 

Did 4-1 

Minor errors 0 

Didn't -1 

h. Configuration Changes: 

Smooth, proper airspeed, small heading and altitude 

deviations 41 

Proper, but 4 5 degrees and 4 50 feet deviations 0 

Rough, busted airspeed, late -1 



50 



8. FINAL APPROACH (GCA) 

a. Power: 

Smooth, small, anticipatory corrections -1-1 

Occasional major throttle adjustments required 0 

Rough, gross throttle adjustments -1 

b. Attitude/Angle of Attack: 

Smooth, on speed, checked weight vs airspeed/angle of 

attack +1 

Proper, occasionally fast or slow 0 

Rough, consistently fast or slow -1 

c. Heading: 

-h 2 degrees +1 

-h 2-5 degrees 0 

-h > 5 degrees -1 

d. Glide Path Control: 

Smooth and on or consistently ’'slightly above/below” +1 

Occasionally "above” or "below," not rough 0 

Rough or erratic or frequently "above" or "below" -1 

e. Altitude : 

Made minimums smoothly within -h 50 ft. -hi 

Rough, or more than 50 ft. above minimums 0 

Busted -1 

f. Missed Approach: 

Smooth, proper, timely -hi 

Proper, minor pitch/bank deviations 0 

Rough, overbanked, late, overcontrolled -1 
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9. CLIMBOUT AND DEPARTURE 

a. Climb Schedule: 

+ 10 KIAS, smooth +1 

+ 15 KIAS 0 

+ 15 more than once, or rough -1 

b. Adherence to Departure Clearance: 

No errors (+ 5 degrees), (+ 100 feet) +1 

Minor errors (5-10 degrees), (+ 200 feet) 0 

Major error (s) (busted clearance, missed check point) -1 

c. Climb Safety Checks: 

Done, correctly, timely +1 

Done, perfunctorily 0 

Missed or mistaken -1 

d. Navaid Usage: 

Checked ID, correct selection, IFF +1 

Minor errors, but made proper path 0 

No ID check, selectors wrong, missed freq. change -1 

e. Radial Course Control During Transition Climb: 

+ 2 degrees +1 

+ 2-5 degrees 0 

+ > 5 degrees -1 

f. Arrival at Assigned Altitude/Fl: 

+ 100 ft. and smooth +1 

100-200 ft. 0 

+ > 200 ft. or rough -1 

g. Voice Radio Procedures: 

Clear, concise, timely, sharp +1 

Proper 0 

Confused, incorrect or untimely -1 
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10. RETURN TO MONTEREY (check one) IFR [7.J VFR j ] 

a. Weather Check: 

Did, received and interpreted correctly +1 

Minor error, or relied on previous forecast 0 

Failed to consider -1 

b. IFR Cancellation: 

Proper voice procedure, IFF, VFR altitude -FI 

Minor errors, or not applicable 0 

Uncertain, improper procedure -1 

c. Navigation: 

Certain, smooth, correct -Fl 

Wandered somewhat, but proceeded purposefully 0 

Aimless, confused, violated restricted area, etc. -1 

d. Cruise Control: 

Altitude, airspeed, fuel and oxygen OK -Fl 

Minor inattention or minor error 0 

Wasted fuel or 0^, wrong altitude, behind -1 

e. Basic Airwork: 

Very smooth and positive attitude control -Fl 

Reasonably smooth, maintained adequate control 0 

Rough, erratic, uncoordinated movements or behind -1 
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11. ENTRY IFR (check if applicable) PD 

a. Headwork and Anticipation, Descent Checklist: 

Had previewed, fuel/weather checks, planned descent +1 

Responded properly, but not well ahead, and not behind 0 

No anticipation, got behind -1 

b. Basic Airwork: 

Very smooth, always in control +1 

Generally smooth 0 

Improper corrections, rough, or both -1 

c. Radial/Arc Course Control: 

+ 2 degrees and + 1 mile, occasional deviation -+1 

+ 2-5 degrees and + 1-2 miles, occasional deviations 0 

Frequent > + 5 degrees and/or > 2 miles -1 

d. Arrival at Assigned Altitude: 

+ 100 ft. and smooth +1 

100-200 ft. 0 

+ > 200 ft. or rough -1 

e. Voice Radio Procedures: 

Clear, concise, timely, sharp +1 

Proper 0 

Confused, incorrect or untimely -1 

f. Basic Airwork: 

Very smooth, always in control +1 

Generally smooth 0 

Improper corrections, rough, or both -1 

g. Clearance Adherence: 

Did +1 

Minor errors 0 

Didn 1 t -1 

h. Configuration Changes: 

Smooth, proper airspeed, small heading and altitude 

deviations +1 

Proper, but + 5 degrees and + 50 feet deviations 0 

Rough, busted airspeed, late -1 
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12 ENTRY VFR (check if applicable) [ j 

a. Check Points/Altitudes/Airspeeds: 

Knew and complied with course rules, on speeds and altitude +1 

No violations 0 

Didn't know, or missed, points, altitudes and/or speeds -1 

b. Voice Procedures: 

Clear, concise, correct +1 

Adequate 0 

Arrogant, abusive, or interfered with traffic -1 

c. Headwork: 

Had previewed, fuel/weather checks, planned descent +1 

Responded properly, but not well ahead, and not behind 0 

No anticipation, got behind -1 

d. Lookout Doctrine: 

Head up and out, respected traffic constraints +1 

Adequate, no interference with traffic 0 

Didn't look, or interfered -1 

e. Adherence to Clearances: 

Did, in a positive manner, with foresight +1 

Did, perfunctorily 0 

Did not comply with (a) clearance (s) -1 



55 



13. IFR APPROACH 

a. Power: 

Smooth, small, anticipatory corrections +1 

Occasional major throttle adjustments required 0 

Rough, gross throttle adjustments -1 

b. Attitude/Angle of Attack: 

Smooth, on speed, checked weight vs airspeed/angle of 

attack +1 

Proper, occasionally fast or slow 0 

Rough, consistently fast or slow -1 

c. Heading: 

+ 2 degrees +1 

+ 2-5 degrees 0 

+ > 5 degrees -1 

d. Glide Path Control: 

Smooth and on or consistently "slightly above/below" +1 

Occasionally "above" or "below," not rough 0 

Rough or erratic or frequently "above" or "below" -1 

e. Altitude: 

Made minimums smoothly within + 50 feet +1 

Rough, or more than 50 feet above minimums 0 

Busted -1 

f. Missed Approach: 

Smooth, proper, timely +1 

Proper, minor pitch/bank deviations 0 

Rough, overbanked, late, overcontrolled -1 
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14. FULL STOP LANDING 

a. Airspeed, Angle of Attack, Use of All-At t i tude-Indicator : 

Smooth, on speed, checked weight vs airspeed/angle of 

attack +1 

Proper, occasionally fast or slow 0 

Rough, consistently fast or slow -1 

b. Power Control: 

Smooth, small, anticipatory corrections +1 

Occasional major throttle adjustments required 0 

Rough, gross throttle adjustments -1 

c. Basic Airwork: 

Very smooth, always in control +1 

Generally smooth 0 

Improper corrections, rough, or both -1 

d . Checklis t : 

Knew where to find and how to use desired information +1 

Knew what he wanted but not where to find it 0 

Failed to use or misinterpreted pubs -1 

e. Touchdown Technique: 

Proper, smooth, level bank, on speed (or nice flare) +1 

Respectable, + 5 KIAS 0 

Rough, erratic, 10 or more KIAS fast, or landed long -1 

f. Rollout Heading/Alt itude Control: 

Smooth, proper use of rudder, brakes, steering and 

aerodynamic braking +1 

Adequate and safe, but not impressive, no damaged tire, 

no hot brakes, didn f t oversteer 0 

Excessive braking, steering or swerve, or no aerodynamic 

braking -1 

g. Headwork (Data Recorders judgment, examples given): 

Used good sense and judgment (waved off bad approach) +1 

OK 0 

Misjudged, too aggressive, or too complacent (tried to 

salvage bad approach) -1 

h. Turnoff/Taxi Technique: 

Smooth, properly slow, correct configuration +1 

Rough but adequate, complied with procedures 0 

Fast, improper configuration, improper power/brake -1 

i. Voice Procedures: 

Timely switch, proper calls, complied with clearance, 

did not interfere with tower/ground control +1 

Adequate 0 

Late or didn't switch, poor procedure, or interfered -1 
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15. SHUTDOWN 

a. Parking: 

Proper brake/power usage, cooperated with director 
Some power/brake problems, but adequate 
Rough, fast, uncooperative, misjudged turn 

b. Systems Shutdown/Checklist: 

Used checklist, proper procedure, orderly, complete 
Complete but random 
Incomplete or misused 

c. Engine Shutdown: 

Timely compliance, anticipated blowout (T-l) 

Delayed, but proper compliance 
Late, non-compliance, or indecision 

d. Cockpit Cleanup (check pilot inspect): 

Thorough and adequate 

Missed one or two minor switches 

Major, or more than two switches wrong 

e. Postflight Inspection: 

Did, thoroughly 
Perfunctory 

Didn’t, or missed discrepancy 

16. GENERAL CATEGORIES 

a. Yellow Sheet: 

Correct, brief, legible 

Adequate, needed help 

Error, omission, illegible or sloppy 

b. Handling of Emergencies/Malfunctions (if applicable) : 

Did, proper procedures, good judgment 

Slow reaction, but correct; or not applicable 

Error, indecisive, lack of knowledge 

c. Pilot’s Personal Equipment: 

All on, clean, and in good shape 
Complete, but not smart 
Gear missing or very dirty 

d. Airplane Condition (no grade, check box) 

Outstanding 

Average and adequate 

Airplane condition was such that it might bias pilot’s score 
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APPENDIX D 

DATA FLIGHT SUMMARY DATA 
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AVIATOR FLIGHT EVALUATION DATA FLIGHT PHASE SCORES 
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Phases Total Total Total 

8 9 10 11 12 13 14 15 16 (+) (-) Net Average 
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Phases Total Total Total 

8 9 10 11 12 13 14 15 16 (+) (-) Net Average 
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Phases Total Total Total 

7 8 9 10 11 12 13 14 15 16 (+) (-) Net Average 
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AVIATOR FLIGHT EVALUATION DATA AGGREGATED CATEGORY SCORES 
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*Prop.: proportions of possible scores normalized using the sum of the maximum possible positive 

score in each subcategory, 1 through 16. 
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0.359 



*Prop.: proportions of possible scores normalized using the sum of the maximum possible positive 

score in each subcategory, 1 through 16. 
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250 



*Prop.: proportions of possible scores normalized using the sum of the maximum possible positive 

score in each subcategory, 1 through 16 . 
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*Prop.: proportions of possible scores normalized using the sum of the maximum possible positive 

score in each subcategory, 1 through 16. 
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